Image Similarity Search

Image similarity search combines vector search with graph traversal. A descriptor represents an image as a vector; a connection links that descriptor to the stored image. A query finds nearby descriptors and follows their connections to retrieve the corresponding images.

This example uses five JPEGs and four-dimensional synthetic vectors. The vectors encode sample order, so their distances illustrate the query flow without measuring visual similarity. All input images are provided below. The image similarity search application covers vectors generated from image content by a model.

Ingest Images and Descriptors into ApertureDB

A descriptor set defines the vector dimensions, search engine, and distance metric. This request creates a four-dimensional set using HNSW and L2. The name docs_similarity_example must be unused; an existing set requires a different name throughout the example.

[{
    "AddDescriptorSet": {
        "name": "docs_similarity_example",
        "dimensions": 4,
        "engine": "HNSW",
        "metric": "L2"
    }
}]

Each descriptor and its image share a sample_id property. The following query inserts sample 0 and connects the image to the descriptor through has_descriptor. Supply two input blobs in command order: the vector [0.0, 1.0, 0.0, 0.0] encoded as four float32 values, then the JPEG bytes. See blob arrays for the wire representation.

[{
    "AddDescriptor": {
        "set": "docs_similarity_example",
        "properties": {
            "sample_id": 0
        },
        "_ref": 1
    }
}, {
    "AddImage": {
        "format": "jpg",
        "properties": {
            "sample_id": 0,
            "doc_fixture": "image_similarity_example"
        },
        "connect": {
            "class": "has_descriptor",
            "ref": 1
        }
    }
}]

The remaining samples use the same insertion pattern with sample IDs 1 through 4 and the inputs below. The doc_fixture property identifies the images belonging to this example. Each JPEG is available as a download.

Sample images and vectors

sample_id	Vector	JPEG
0	[0, 1, 0, 0]	input.jpg
1	[1, 1, 0, 0]	result_0.jpg
2	[2, 1, 0, 0]	result_1.jpg
3	[3, 1, 0, 0]	result_2.jpg
4	[4, 1, 0, 0]	result_3.jpg

Query for Similar Images

Pass sample 0’s descriptor, [0, 1, 0, 0], as the single input blob. k_neighbors: 5 selects the source and all four neighbors. The descriptor _ref lets the second command retrieve their connected images. The image constraint excludes sample 0, and blobs: true returns the other JPEGs. Images are explicitly sorted by sample_id; their order does not implicitly inherit descriptor distance order.

[{
    "FindDescriptor": {
        "_ref": 1,
        "set": "docs_similarity_example",
        "k_neighbors": 5,
        "distances": true,
        "results": {
            "list": ["sample_id"]
        }
    }
}, {
    "FindImage": {
        "is_connected_to": {
            "ref": 1
        },
        "constraints": ["$sample_id", "!=", 0],
        "results": {
            "list": ["sample_id"]
        },
        "sort": {
            "key": "sample_id",
            "order": "ascending"
        },
        "blobs": true
    }
}]

For these vectors, L2 reports squared Euclidean distances of 0, 1, 4, 9, and 16. Each _blob_index identifies an image in the returned blob array.

[{
    "FindDescriptor": {
        "entities": [{
            "_distance": 0.0,
            "sample_id": 0
        }, {
            "_distance": 1.0,
            "sample_id": 1
        }, {
            "_distance": 4.0,
            "sample_id": 2
        }, {
            "_distance": 9.0,
            "sample_id": 3
        }, {
            "_distance": 16.0,
            "sample_id": 4
        }],
        "returned": 5,
        "status": 0
    }
}, {
    "FindImage": {
        "blobs_start": 0,
        "entities": [{
            "_blob_index": 0,
            "sample_id": 1
        }, {
            "_blob_index": 1,
            "sample_id": 2
        }, {
            "_blob_index": 2,
            "sample_id": 3
        }, {
            "_blob_index": 3,
            "sample_id": 4
        }],
        "returned": 4,
        "status": 0
    }
}]

The four returned images appear in sample_id order:

With model-generated embeddings, stored and query vectors share the same model and preprocessing. The descriptor set’s dimensions and distance metric must match that model. A useful relevance threshold depends on those embeddings; the synthetic distances above do not define one.

The example images can be removed with a DeleteImage constraint on doc_fixture == "image_similarity_example". A subsequent DeleteDescriptorSet removes docs_similarity_example and its descriptors. Deleting the images also removes their connections.